Probabilistic Hill-climbing

Many learning tasks involve searching through a discrete space of performance elements, seeking an element whose future utility is expected to be high. As the task of nding the global optimum is often intractable, many practical learning systems use simple forms of hill-climbing to nd a locally optimal element. However, hill-climbing can be complicated by the fact that the utility value of a performance element can depend on the distribution of problems, which typically is unknown. This paper formulates the problem of performing hill-climbing search in settings where the required utility values can only be estimated on the basis of their performance on random test cases. We present and prove correct an algorithm that returns a performance element that is arbitrarily close to a local optimum with arbitrarily high probability. Council of Canada. All three author gratefully acknowledge receiving many helpful comments from David Mitchell and the anonymous reviewers.